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Elliot  E.  Entin,  Frederick  J.  Diedrich,  Jean  MacMillan,  &  Daniel  Serfaty 

Aptima®,  Inc.1 

Abstract 

In  this  paper  we  test  the  hypothesis  that  organizational  structure,  and  the  history  of  that 
structure,  influences  mutual  workload  awareness.  More  specifically,  we  explored  the 
congruence  of  workload  awareness  among  a  number  of  decision  makers  acting  in  two 
organizational  structures  (functional  vs.  divisional)  with  different  histories  (divisional 
followed  by  functional  vs.  functional  followed  by  divisional).  Seven  teams  comprised  of 
military  officers  were  assigned  to  one  of  the  two  orders  and  performed  a  simulated 
mission.  Findings  show  that  workload  awareness  was  higher  in  the  functional-divisional 
than  in  the  divisional-  functional  order  indicating  that  workload  awareness  can  be 
influenced  by  factors  such  as  organizational  structure  and  how  that  structure  changes  over 
time.  There  is  also  evidence  that  high  workloads  may  foster  higher  workload  awareness, 
and  that  high  workload  awareness  may  ameliorate  some  of  the  negative  effects  of  high 
workload. 

Introduction 

In  his  discussion  of  network  centric  warfare  and  related  concepts,  ADM  Cebrowski 
describes  self-synchronization  as:  “The  ability  of  a  well-informed  force  to  organize  and 
synchronize  complex  warfare  activities  from  the  bottom  up.  The  organizing  principles  are 
unity  of  effort,  clearly  articulated  commander's  intent,  and  carefully  crafted  rules  of 
engagement  (ROE).  Self-synchronization  is  enabled  by  a  high  level  of  knowledge  of 
one's  own  forces,  enemy  forces,  and  all  appropriate  elements  of  the  operating 
environment.  It  overcomes  the  loss  of  combat  power  inherent  in  top-down,  command- 
directed  synchronization  characteristic  of  more  conventional  doctrine  and  converts 
combat  from  a  step  function  to  a  high-speed  continuum.” 

As  ADM  Cebrowski  states,  to  foster  self-synchronization  a  commander  must  have  a  high 
level  of  awareness  of  the  battle  space  and  the  intentions  and  actions  of  other 
commanders.  However,  the  factors  that  influence  such  situational  and  mutual  awareness 
are  not  completely  understood.  Indeed,  both  the  accuracy  and  need  for  awareness  of 
others  may  depend  on  the  structure  of  the  organization  to  which  the  commanders  belong 
and  the  tasks  that  they  are  trying  to  perform.  Moreover,  awareness  may  be  affected  by 
the  history  of  the  organization. 

There  is  a  sizeable  literature  regarding  the  mediating  effects  of  awareness  and  particularly 
situational  awareness  on  performance.  For  instance,  Endsley  (e.g.,  1988,  1995a,  1995b) 
has  presented  a  compelling  argument  linking  situational  awareness  and  performance.  She 
notes  that  obtaining  and  maintaining  good  situational  awareness  often  occupies  a  major 
portion  of  an  operator’s  role.  Operators  of  large  system  such  as  flexible  manufacturing 
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systems  must  constantly  revise  their  knowledge  of  system  parameters  to  keep  the  system 
operating  effectively.  Similarly,  within  tactical  and  strategic  systems  military 
commanders  depend  on  situational  awareness  as  a  precursor  to  decision  making.  When 
awareness  is  poor,  as  it  was  in  the  U.S.S.  Vincennes  incident,  the  consequences  can  be 
dire.  In  the  operational  assessment  of  the  AH-64D  (Longbow)  helicopter,  improved 
performance  was  attributed  to  increased  situational  awareness  (Eidelkind,  Moffett, 
Arendt,  and  McKee,  1995).  Likewise,  in  a  review  of  18  AH-64  accidents  that  had 
occurred  in  an  8-month  time  period  (Entin  and  Zeller,  1997),  investigators  found  that  in 
nine  of  them  a  loss  of  situational  awareness  by  one  or  both  crewmembers  may  have 
contributed  to  the  accident.  It  appears  that  situational  awareness  can  have  a  profound 
effect  on  performance. 

The  concept  of  shared  mental  models  also  suggests  a  key  role  for  situational  awareness 
(Stout  et  al.,  1999;  Cannon-Bowers,  Salas,  and  Converse,  1993,  Rouse,  Cannon-Bowers, 
and  Salas,  1992).  The  essential  idea  is  that  shared  mental  models:  “provide  team 
members  with  a  common  understanding  of  who  is  responsible  for  what  task  and  what  the 
information  requirements  are”  (Stout  et  al.,  1999,  p.  61).  The  implication  is  that  this 
common  understanding  serves  to  coordinate  performance  and  is  an  important  factor  of 
good  team  performance  outcome. 

In  our  paper  we  address  the  awareness  team  members  have  of  each  other.  More 
specifically,  the  awareness  of  each  other’s  workload:  knowing  what  other  team  members 
around  you  are  doing  and  how  involved  they  are  is  a  part  of  situational  awareness.  It  also 
contributes  to  good  teamwork  skills  (Glickman  and  Zimmer,  1989;  Glickman  et  al., 
1987).  For  instance,  monitoring  and  backup  behavior  are  two  important  teamwork  skills 
(Glickman  et  al.,  1987)  and  both  are  linked  to  an  awareness  of  other  team  member’s 
workload  conditions.  Awareness  of  other  team  member’s  workload  enhances  general 
situational  awareness  and  provides  information  team  members  can  use  to  manage 
workload  balance  within  the  team.  Team  members  who  are  overwhelmed  can  be  offered 
help  and  workload  can  be  shifted  to  under  loaded  team  members.  Such  management  of 
workload  balance  can  improve  overall  team  performance. 

Shared  awareness  and  workload,  however,  may  be  mediated  by  organizational  structure. 
For  example,  performance  outcome  and  process  measures  were  significantly  higher  in  a 
non-traditional  command  and  control  (C  )  architecture  optimized  for  the  mission  than  a 
traditional  C"  architecture  that  was  not  optimized  for  the  mission  (Entin,  1999).  Entin 
(1999)  concluded  that  various  elements  of  the  non-traditional  optimized  C  architecture 
facilitated  several  underlying  team  processes  that  in  turn  facilitated  performance. 
Hollenbeck  et  al.  (1999)  and  Moon  et  al.  (2000)  have  also  showed  that  team 
organizational  structure  mediates  team  performance.  Moreover,  Moon  et  al.  (2000) 
reported  that  performance  was  higher  for  teams  that  transitioned  from  a  functional  to 
divisional  structure  compared  to  teams  that  transitioned  from  a  divisional  to  functional 
structure.  Generally  speaking,  a  divisional  structure  provides  each  entity  of  the 
organization  with  a  mix  of  organizational  resources/assets,  whereas  in  a  functional 
structure  each  entity  is  specialized  to  meet  a  specific  need  and  owns  the  specific  class  of 
assets  to  do  so. 


Study  Objectives 

In  this  study,  we  asked  how  a  team’s  structure,  and  the  history  of  that  structure, 
influenced  workload  awareness.  More  specifically,  we  explored  the  congruence  of 
workload  awareness  among  a  number  of  decision  makers  acting  in  two  organizational 
structures  (functional  vs.  divisional)  with  different  histories  (divisional  then  functional  vs. 
functional  then  divisional).  To  assess  workload  awareness,  we  computed  the  deviation 
between  what  a  team  member  said  his/her  workload  was  to  what  each  of  the  other  team 
members  thought  it  was.  We  expected  workload  and  workload  awareness  to  be 
differentially  affected  by  the  structural  manipulations.  We  also  expect  a  relationship 
between  workload  awareness  and  performance. 

Our  observations  of  workload  awareness  and  organizational  structures  were  part  of  a 
larger  methodological  study  carried  out  to  examine,  among  other  things,  the  elements  that 
are  necessary  to  create  divisional  and  functional  C"  organizational  structures.  We  will 
briefly  review  the  methodology  used,  but  for  a  more  detailed  description,  see  Diedrich  et 
al.  (2002,  this  volume). 


Method 


Participants 

Forty-two  officers  attending  the  Naval  Postgraduate  School  served  as  participants.  Most 
of  the  officers  were  03s  or  04s  and  several  services  were  represented.  Participants  were 
organized  into  seven  teams  of  six  individuals  each. 

Experimental  Design  and  Independent  Variables 

Organizational  structure  was  varied  as  a  within-subjects  factor  and  contrasted  a 
functional  organization  where  each  participant  specialized  in  one  aspect  of  the  mission 
such  as  air  warfare  with  a  divisional  organization  where  each  participant  was  tasked  to 
perform  several  aspects  of  the  mission.  In  the  functional  organizational  structure,  a 
participant  charged  with  one  aspect  of  the  mission,  such  as  air  warfare,  owned  the  assets 
necessary  to  do  that  part  of  the  mission  even  if  those  assets  were  on  various  platforms. 
When  participants  performed  in  the  divisional  organizational  structure  their  platform  was 
multifunctional  and  to  some  extent  could  process  air,  surface,  and  subsurface  treats. 

Order  of  presentation  was  counterbalanced  across  teams:  half  of  the  teams  started  in 
divisional  followed  by  functional  organizational  structure  and  half  started  in  functional 
followed  by  divisional  organizational  structure. 

Dependent  Measures 

The  Task  Load  Index  (TLX;  Hart  &  Staveland,  1988)  was  used  to  assess  individual  team 
members’  workload.  We  have  extended  the  TLX  to  capture  workload  awareness  as  well 
(Entin,  Serfaty,  and  Kerrigan,  1998).  In  the  first  part  of  a  two-part  questionnaire  team 
members  reported  their  own  workload  in  terms  of  the  traditional  items  comprising  the 
TLX  (i.e.,  mental  effort,  time  pressure,  performance,  effort,  and  frustration).  In  the 
second  part  of  the  questionnaire  each  participant  provided  an  estimate  of  the  overall 
workload  experienced  by  each  of  the  other  team  members.  Throughout  the  instrument 
responses  were  made  to  a  21 -point  scale  anchored  at  one  end  by  the  word  low  and  at  the 


end  by  the  word  high.  Workload  awareness  was  computed  by  comparing  each  team 
member’s  estimate  of  a  particular  team  member’s  workload  to  the  particular  team 
member’s  self-rating.  The  deviations  of  the  estimates  from  the  self-rating  were  squared, 
averaged,  and  the  square  root  taken  to  arrive  at  the  root  mean  square  deviation  (RMS- 
deviation).  RMS-deviation  was  the  measure  of  workload  awareness.  The  smaller  the 
RMS-deviation,  the  less  each  team  member’s  estimate  deviates  from  the  self  workload 
rating,  thus  the  greater  the  awareness.  Conversely,  the  larger  the  RMS-deviation,  the 
more  each  team  member’s  estimate  deviates  from  the  self  workload  rating,  thus  the  lower 
the  awareness. 

The  simulator  hosting  the  experiment  was  capable  of  capturing  most  of  a  team’s  actions. 
Of  particular  interest  were  the  various  tasks  comprising  the  mission.  The  performance 
measure  for  most  of  the  tasks  involved  counts,  such  as  the  number  of  SCUD  launchers 
destroyed,  the  number  of  SCUD  missiles  shot  down,  or  number  of  enemy  ships 
destroyed.  Performance  scores  were  computed  for  18  such  tasks. 

Simulator  and  Scenario 

The  study  was  hosted  on  the  Dynamic  Distributed  Decision  Making  (DDD)  simulator. 
Most  experimental  work  within  the  A2C2  project  has  been  performed  employing  the 
moderate  fidelity  DDD  research  simulator  (Kleinman,  Young,  and  Higgins,  1996).  The 
DDD  has  been  used  extensively  since  1989  in  research  involving  “open  ocean”  naval 
team  decision-making.  It  has  served  as  the  vehicle  by  which  teams  of  subjects  interact  in 
a  dynamically  evolving  tactical  scenario.  The  DDD  allows  for  a  high  degree  of 
experimental  control  and  provides  on-line  data  collection  of  subjects’  interactions  in  a  log 
file  that  can  be  used  to  develop  performance  variables. 

The  scenario,  a  collaborative  effort  of  the  Naval  Postgraduate  School,  Aptima,  and  the 
University  of  Connecticut,  depicts  a  mission  that  involved  preparing  a  battlespace  for  the 
arrival  of  follow-on  forces  by  engaging  hostile  assets  perceived  as  immediate  threats 
(e.g.,  aircraft,  patrol  boats,  destroyers,  and  submarines).  In  addition,  players  were 
charged  with  destroying  an  enemy  air  base  and  an  enemy  naval  base,  each  heavily 
guarded  by  SAMs,  and  with  defending  neighboring  foreign  friendly  areas  from  SCUD 
missile  attacks.  An  example  of  a  typical  display  is  shown  in  Fig.  1.  The  white  areas  in 
the  figure  represent  water,  the  shaded  areas  represent  land,  and  areas  with  heavy  boarders 
are  the  areas  Blue  is  to  protect  from  Red  (the  enemy).  The  area  to  the  right  of  the  map  is 
the  control  area.  Important  to  the  workload  awareness  issue,  many  tasks  require  the 
coordination  of  different  assets  that  frequently  are  controlled  by  different  team  members. 
Some  tasks  are  time  critical  (e.g.,  SCUD  missile  launchers,  coastal  defense  launchers) 
and  thus  require  quick  response  for  effective  prosecution. 

Procedure 

After  being  trained  in  the  use  of  the  DDD  simulation,  each  team  completed  two  practice 
trials  and  a  data  collection  trial  for  one  structure  followed  several  days  later  with  two 
practice  trials  and  a  data  collection  trial  for  the  other  structure. 


Figure  1.  Typical  Scenario  Display 


Results  and  Discussion 

The  self-perception  of  workload  for  each  team  member  was  averaged  to  form  a  mean 
workload  for  each  team.  As  we  can  see  from  Fig.  2,  perceived  workload  was 
significantly  higher  in  the  functional  than  in  the  divisional  organizational  structure,  (p  < 
0.04;  means  for  the  divisional  and  functional  were  10.8  and  11.6,  respectively).  Figure  3 
shows  that  workload  was  perceived  to  be  higher  for  the  functional  than  divisional 
organizational  structures  by  just  about  every  team.  These  results  indicate  that 
participants  perceived  more  work  was  required  to  perform  the  mission  with  a  functional 
architecture.  This  is  a  reasonable  finding  given  that  the  functional  architecture  engenders 
a  global  view,  whereas,  the  divisional  architecture  tends  to  foster  a  more  local 
perspective.  These  results  also  imply  that  workload  awareness  might  be  more  important 
for  the  functional  organizational  structure  due  to  higher  workloads  where  some  team 
members  might  require  back-up  assistance. 

The  analysis  of  workload  awareness,  however,  revealed  no  overall  differences  in 
awareness  between  the  divisional  and  functional  organizational  structures.  These  results 
suggest  that  higher  workloads  do  not  correspond  to  heightened  workload  awareness.  Yet, 
the  results  discussed  below  suggest  this  might  not  be  the  case. 


Figure  2.  Mean  Workload  For  The  Divisional  and  Functional  Organizational  Structures 
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Figure  3.  Workload  For  Each  Team  By  Organizational  Structure 

Figure  4  presents  the  mean  RMS -deviation  for  the  divisional-functional  and  functional- 
divisional  orders.  These  data  indicate  that  awareness  tended  to  be  higher  (i.e.,  RMS- 
deviation  smaller)  in  the  functional-divisional  than  divisional-functional  order  (p  <  .07). 
Examining  further  we  see  that  at  Trial  1  the  awareness  of  other’s  workload  was  superior 
in  the  functional  than  in  the  divisional  organizational  structure  (p  <  .05).  At  Trial  2 
workload  awareness  appeared  to  improve  for  both  orders,  however,  more  so  for  the 
divisional-functional  than  functional-divisional  order,  thereby  erasing  the  difference  that 
had  existed  at  Trial  1. 

To  seek  an  explanation  for  these  results  we  examined  the  perceived  workload  in  Trial  1 
and  Trial  2.  We  suspected  that  heightened  workload  did  foster  higher  workload 
awareness,  and  analysis  of  the  perceived  workload  for  Trial  1  showed  that  workload  was 
significantly  higher  under  the  functional  than  divisional  organizational  structures  (p  < 
.05).  The  same  analysis  for  Trial  2  revealed  an  increase  in  workload  for  the  divisional 
and  slight  decrease  for  the  functional  structures,  hence  no  statistically  significant 
difference  was  observed.  Thus,  an  explanation  for  the  observed  pattern  of  results  is  that 
teams  who  had  the  functional  condition  first  (Trial  1)  perceived  higher  workloads  and 
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thus  were  more  aware  of  their  teammates  than  teams  that  had  the  divisional  condition 
first.  These  teams  then  stayed  at  a  similar  level  of  awareness  as  they  went  into  the 
divisional  condition  in  Trial  2.  In  contrast,  since  the  perceived  workload  was  less  for 
teams  that  started  out  in  a  divisional  structure  they  were  initially  less  aware  of  others 
because  there  was  not  as  great  a  need  to  be  aware.  These  teams,  however,  had  to  play 
"catch-up"  when  they  went  to  the  higher  workload  environment  of  the  functional 
organizational  structure  in  Trial  2. 


Figure  4.  Workload  Awareness  As  Measured  By  RMS  Deviation  For  The  Divisional- 
Functional  And  Functional-Divisional  Orders  By  Trial 

Next,  we  examined  performance  by  considering  all  the  tasks  (i.e.,  enemy  entities)  a  team 
could  attack  as  sanctioned  by  the  standing  rules  of  engagement.  Those  tasks  included,  for 
example,  coastal  defense  missiles,  SCUDs,  and  SCUD  launchers.  For  each  task  the 
number  of  attacks  were  normalized  by  the  number  that  could  be  attacked,  thus  an  attack 
ratio  of  0.75  means  that  three-quarters  of  the  tasks  that  could  be  attacked  were  attacked. 
Performance,  in  terms  of  mean  attack  ratio,  is  depicted  for  the  divisional  and  functional- 
organizational  structures  in  Fig.  5.  There  were  no  statistical  differences  between 
performances  in  the  two  organizational  structures.  However,  as  shown  above,  workload 
was  higher  in  the  functional  than  in  the  divisional  structure,  and  we  know  that 
performance  is  usually  negatively  related  to  workload  (see  for  example  Entin,  1999). 
Why  then  were  there  no  performance  differences  between  the  two  organizational 
structures?  We  surmise  that  the  higher  workload  awareness  in  the  functional 
organizational  structure  helped  ameliorate  the  negative  effects  of  high  workload. 

Conclusion 

Collectively,  these  data  show  that  workload  awareness  can  be  influenced  by  factors  such 
as  organizational  structure  and  how  that  structure  changes  over  time.  Workload 
awareness  was  found  to  be  higher  in  a  functional-divisional  ordering  of  organizational 
structures  than  in  the  divisional-functional  ordering.  Hollenbeck  et  al  (1999)  have 
surmised  that  communication  among  team  members  is  facilitated  more  by  a  functional 
than  a  divisional  structure.  The  greater  exchange  of  information  initiated  by  the 
functional  structure  may  partially  explain  the  higher  workload  awareness  observed  in  the 
functional-divisional  order.  There  is  also  evidence  that  high  workloads  may  foster  higher 
workload  awareness  and  that  high  workload  awareness  may  ameliorate  some  of  the 
negative  effects  of  high  workload.  We  must,  however,  keep  in  mind  that  the  reported 


workload  for  the  conditions  studied  here  was  in  the  moderate  range  at  best.  As  workload 
increases  to  higher  levels,  these  observations  may  no  longer  hold. 


Organizational  Structure 


Figure  5.  Performance  For  The  Divisional  And  Functional  Organizational  Structures 

If  we  can  assume  that  workload  awareness  is  an  important  component  of  situational 
awareness,  than  it  is  reasonable  to  assume  the  situational  awareness  may  be  impacted  by 
organizational  structure  and  how  that  structure  evolves  over  time.  Indeed,  since  accurate 
awareness  provides  a  foundation  for  self-synchronization,  these  data  imply  that  efforts  to 
foster  self-synchronization  and  coordination  among  commanders  will  necessarily  be 
influenced  by  a  myriad  of  factors  associated  with  the  command  environment.  The  work 
reported  here  represents  initial  efforts  to  define  how  some  of  these  factors  interrelate. 
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